How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy

ML models are ubiquitous in real world applications and a constant focus of research. At the same time, community has started to realize importance protecting privacy training data. Differential Privacy (DP) become gold standard for making formal statements about data anonymization. However, while some adoption DP happened industry, attempts apply complex still few far between. The is hindered by limited practical guidance what protection entails, guarantees aim for, difficulty achieving good privacy-utility-computation trade-offs models. Tricks tuning maximizing performance scattered among papers or stored heads practitioners. Furthermore, literature seems present conflicting evidence on how whether architectural adjustments which components "safe" use with DP. This work self-contained guide that gives an in-depth overview field presents information best possible model rigorous guarantees. Our target audience both researchers Researchers interested will benefit from clear current advances areas improvement. We include theory-focused sections highlight important topics such as accounting its assumptions, convergence. For practitioner, we provide background theory step-by-step choosing appropriate definition approach, implementing training, potentially updating architecture, hyperparameters. practitioners, consistently fully reporting critical, so propose set specific practices stating